Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology

Stain variation is a phenomenon observed when distinct pathology laboratories stain tissue slides that exhibit similar but not identical color appearance. Due to this color shift between laboratories, convolutional neural networks (CNNs) trained with images from one lab often underperform on unseen images from the other lab. Several techniques have been proposed to reduce the generalization error, mainly grouped into two categories: stain color augmentation and stain color normalization. The former simulates a wide variety of realistic stain variations during training, producing stain-invariant CNNs. The latter aims to match training and test color distributions in order to reduce stain variation. For the first time, we compared some of these techniques and quantified their effect on CNN classification performance using a heterogeneous dataset of hematoxylin and eosin histopathology images from 4 organs and 9 pathology laboratories. Additionally, we propose a novel unsupervised method to perform stain color normalization using a neural network. Based on our experimental results, we provide practical guidelines on how to use stain color augmentation and stain color normalization in future computational pathology applications.Comment: Accepted in the Medical Image Analysis journa

Improving tumor budding reporting in colorectal cancer : a Delphi consensus study

Tumor budding is a long-established independent adverse prognostic marker in colorectal cancer, yet methods for its assessment have varied widely. In an effort to standardize its reporting, a group of experts met in Bern, Switzerland, in 2016 to reach consensus on a single, international, evidence-based method for tumor budding assessment and reporting (International Tumor Budding Consensus Conference [ITBCC]). Tumor budding assessment using the ITBCC criteria has been validated in large cohorts of cancer patients and incorporated into several international colorectal cancer pathology and clinical guidelines. With the wider reporting of tumor budding, new issues have emerged that require further clarification. To better inform researchers and health-care professionals on these issues, an international group of experts in gastrointestinal pathology participated in a modified Delphi process to generate consensus and highlight areas requiring further research. This effort serves to re-affirm the importance of tumor budding in colorectal cancer and support its continued use in routine clinical practice.Peer reviewe

Residual cyclegan for robust domain transformation of histopathological tissue slides

Variation between stains in histopathology is commonplace across different medical centers. This can have a significant effect on the reliability of machine learning algorithms. In this paper, we propose to reduce performance variability by using-consistent generative adversarial (CycleGAN) networks to remove staining variation. We improve upon the regular CycleGAN by incorporating residual learning. We comprehensively evaluate the performance of our stain transformation method and compare its usefulness in addition to extensive data augmentation to enhance the robustness of tissue segmentation algorithms. Our steps are as follows: first, we train a model to perform segmentation on tissue slides from a single source center, while heavily applying augmentations to increase robustness to unseen data. Second, we evaluate and compare the segmentation performance on data from other centers, both with and without applying our CycleGAN stain transformation. We compare segmentation performances in a colon tissue segmentation and kidney tissue segmentation task, covering data from 6 different centers. We show that our transformation method improves the overall Dice coefficient by 9% over the non-normalized target data and by 4% over traditional stain transformation in our colon tissue segmentation task. For kidney segmentation, our residual CycleGAN increases performance by 10% over no transformation and around 2% compared to the non-residual CycleGAN. (c) 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license ( http://creativecommons.org/licenses/by/4.0/ )Funding Agencies|Knut and Alice Wallenberg foundationKnut &amp; Alice Wallenberg Foundation</p

Tumor budding t-cell graphs ::assessing the need for resection in pT1 colorectal cancer patients

Colon resection is often the treatment of choice for colorectal cancer (CRC) patients. However, especially for minimally invasive cancer, such as pT1, simply removing the polyps may be enough to stop cancer progression. Different histopathological risk factors such as tumor grade and invasion depth currently found the basis for the need for colon resection in pT1 CRC patients. Here, we investigate two additional risk factors, tumor budding and lymphocyte infiltration at the invasive front, which are known to be clinically relevant. We capture the spatial layout of tumor buds and T-cells and use graph-based deep learning to investigate them as potential risk predictors. Our pT1 Hotspot Tumor Budding T-cell Graph (pT1-HBTG) dataset consists of 626 tumor budding hotspots from 575 patients. We propose and compare three different graph structures, as well as combinations of the node labels. The best-performing Graph Neural Network architecture is able to increase specificity by 20% compared to the currently recommended risk stratification based on histopathological risk factors, without losing any sensitivity. We believe that using a graph-based analysis can help to assist pathologists in making risk assessments for pT1 CRC patients, and thus decrease the number of patients undergoing potentially unnecessary surgery. Both the code and dataset are made publicly available

Taking Tumour Budding to the Next Frontier - a Post-ITBCC 2016 Review.

Tumour budding in colorectal cancer, defined as single tumour cells or small clusters containing four tumour cells or less, is a robust and independent biomarker of aggressive tumour biology. Based on published data in the literature, evidence is certainly in favour of reporting tumour budding in routine practice. One important aspect of implementing tumour budding has been to establish a standardized and evidence-based scoring method as has been recommended by the International Tumour Budding Consensus Conference (ITBCC) in 2016. Further developments have aimed at establishing methods of automated tumour budding assessment. A digital approach to scoring tumour buds has great potential to assist in performing an objective budding count, but in analogy to the manual consensus method, must be validated and standardized. The aim of the present review is to present general considerations behind the ITBCC scoring method and a broad overview of the current status quo and challenges faced by automated tumour budding detection methods

Are tumour grade and tumour budding equivalent in colorectal cancer? A retrospective analysis of 771 patients.

BACKGROUND Tumour grade is traditionally considered in the management of patients with colorectal cancer. However, a large body of literature suggests that a related feature, namely tumour budding, may have a more important clinical impact. The aim of our study is to determine the correlation between tumour grade and tumour budding and their impact on patient outcome. METHODS A retrospective collective of 771 patients with colorectal cancer were included in the study. Clinicopathological information included tumour grade (World Health Organisation 2010; G1, G2 and G3) and tumour budding evaluated as BD1, BD2 and BD3 and representing 0-4 buds, 5-9 buds and 10 or more buds per 0.785 mm2, respectively. RESULTS Tumour grade and tumour budding were correlated (p < 0.0001, percent concordance: 33.8%). Of the BD1 cases, 18.1% were of G3. Only two BD3 cases were G1. Both high tumour grade and tumour budding were associated with higher pT, lymph node metastasis, distant metastasis and lymphatic and venous vessel invasion (p < 0.01, all), but only tumour grade was additionally associated with right-sided tumour location and mucinous histology. Higher tumour budding led to worse overall (p = 0.0286) and disease-free survival (p = 0.001), but tumour grade did not. Budding was independent of tumour grade in multivariate analysis. DISCUSSION Tumour grade and tumour budding are distinct features, as recognised by their different clinicopathological associations, reflecting different underlying biological processes. Nonetheless, tumour budding seems to outperform tumour grade in terms of predicting disease-free survival

Deep learning for multi-class semantic segmentation enables colorectal cancer detection and classification in digital pathology images

In colorectal cancer (CRC), artificial intelligence (AI) can alleviate the laborious task of characterization and reporting on resected biopsies, including polyps, the numbers of which are increasing as a result of CRC population screening programs ongoing in many countries all around the globe. Here, we present an approach to address two major challenges in the automated assessment of CRC histopathology whole-slide images. We present an AI-based method to segment multiple (n=14 ) tissue compartments in the H &amp;E-stained whole-slide image, which provides a different, more perceptible picture of tissue morphology and composition. We test and compare a panel of state-of-the-art loss functions available for segmentation models, and provide indications about their use in histopathology image segmentation, based on the analysis of (a) a multi-centric cohort of CRC cases from five medical centers in the Netherlands and Germany, and (b) two publicly available datasets on segmentation in CRC. We used the best performing AI model as the basis for a computer-aided diagnosis system that classifies colon biopsies into four main categories that are relevant pathologically. We report the performance of this system on an independent cohort of more than 1000 patients. The results show that with a good segmentation network as a base, a tool can be developed which can support pathologists in the risk stratification of colorectal cancer patients, among other possible uses. We have made the segmentation model available for research use on .Funding Agencies|Dutch Cancer Society [10602/2016-2]; Alpe dHuZes / Dutch Cancer Society Fund [KUN 2014-7032]; European Unions Horizon 2020 research and innovation programme [825292]</p

Semi-Supervised Learning to Automate Tumor Bud Detection in Cytokeratin-Stained Whole-Slide Images of Colorectal Cancer

Tumor budding is a histopathological biomarker associated with metastases and adverse survival outcomes in colorectal carcinoma (CRC) patients. It is characterized by the presence of single tumor cells or small clusters of cells within the tumor or at the tumor-invasion front. In order to obtain a tumor budding score for a patient, the region with the highest tumor bud density must first be visually identified by a pathologist, after which buds will be counted in the chosen hotspot field. The automation of this process will expectedly increase efficiency and reproducibility. Here, we present a deep learning convolutional neural network model that automates the above procedure. For model training, we used a semi-supervised learning method, to maximize the detection performance despite the limited amount of labeled training data. The model was tested on an independent dataset in which human- and machine-selected hotspots were mapped in relation to each other and manual and machine detected tumor bud numbers in the manually selected fields were compared. We report the results of the proposed method in comparison with visual assessment by pathologists. We show that the automated tumor bud count achieves a prognostic value comparable with visual estimation, while based on an objective and reproducible quantification. We also explore novel metrics to quantify buds such as density and dispersion and report their prognostic value. We have made the model available for research use on the grand-challenge platform

Fully Automated Tumor Bud Assessment in Hematoxylin and Eosin Stained Whole Slide Images of Colorectal Cancer.

Tumor budding (TB), the presence of single cells or small clusters of up to four tumor cells, at the invasive front of colorectal cancer (CRC) is a proven risk factor for adverse outcomes. International definitions are necessary to reduce the interobserver variability. According to the current international guideline, hotspots at the invasive front should be counted in Hematoxylin and Eosin (H&E) stained slides. This is time-consuming and prone to interobserver variability, therefore there is a need for computer-aided diagnosis solutions. In this paper, we report on developing an Artificial Intelligence (AI) based method for detecting tumor budding in H&E-stained whole slide images. We propose a fully automated pipeline to identify the tumor border, detect tumor buds, characterize them based on their number of tumor cells, and produce a TB density map that we use to identify the TB hot spot. The method outputs the TB count in the hotspot as a computational biomarker. We show that the proposed automated TB detection workflow performs on par with a panel of five pathologists at detecting tumor buds, and that the hotspot-based TB count is an independent prognosticator in both the univariate and the multivariate analysis, validated on a cohort of n=981 CRC patients. Computer-aided detection of tumor buds based on deep learning can perform on par with expert pathologists at detection and quantification of tumor buds in H&E-stained colorectal cancer histopathology slides, strongly facilitating the introduction of budding as an independent prognosticator in clinical routine and clinical trials